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ABSTRACT 

Streamlining involves the modification of 
cost-analysis methods to make them more appropriate for the limited 
time and resources, and immediate information needs, of local-level 
program evaluations. It can also enhance the reliability and validity 
of district-level studies. A review of existing texts of cost 
analysis and of studies of traditional cost analysis methods reveals 
their limited usefulness for district-level evaluators and 
strengthens the argument for the use of streamlined methods. This 
guide focuses primarily on two stages of a cost-outcome analysis: 
measurement of prograun costs and measurement of program outcomes. 
Measuring program costs involves: (1) listing program resources; (2) 
identifying resources representing direct cost to the district; and 
(3) determining the cost of resources. The parameters of the program, 
the reliability of "free" resources, as well as present and 
annualized values need to be included in the tot: 1 cost figure, and a 
sensitivity analysis should be conducted. Streamlining the 
measurement of program outlines involves attention to using a 
quasi-experimental design, evaluating a single program, and the use 
of one or more of four cost analysis methods: (1) cost-feasibility; 
(2) cost-utility; (3) cost-effectiveness; and (4) cost-benefit. A 
brief annotated bibliography is appended. (IW) 
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PURPOSE OF THIS COST-OUTCOME GUIDE SERIES 



This guide series was developed in response to ^valuators' 
reported need for information on how to conduct cost-outcome 
analyses. Recent studies on the use of cost-outcome analyses by 
state education agency (SEA) e\aluators (Smith and Smith, 1984} 
and local education agency (LEA) evaluators (Smith, 1984) showed 
that: 

1. 60 percent of all SEAs anticipate requests in the 
next five years to conduct cost-outcome analyses; 

2. 71 percent of metropolitan LEAs anticipate requests 
in the next five years to conduct cost*outcom3 
studies; and 

3* one of the primary impediments to the conduct of 
cost-outcome analyses is the absence of useful 
guides and resources. 

To assist these and other evaluators in conducting 
cost-outcome analyses, three "how-to" guides were developed. 
Although several texts currently exist on cost-outcome analysis, 
they tend to be written in technical iMguage and do not give 
sufficient attention to the collection of outcomes as well as 
costs. The three guides on cost analysis supplement existing 
texts by providing concise, readable explanations on how to 
conceptualize and conduct cost-outcome studies for program 
evaluation . 

The first cost guide (No. 2 in the series) introduces four 
types of cost-outcome analysis, directs the collection of cost 
data, and explains how to select the most appropriate 
cost-outcome analysis to answer an evaluation question. The 
second cost guide (No. 4 in the series) describes how to design 
an outcome study and outlines the procedures for collecting 
outcome data for cost-utility, cost-effectiveness, and 
cost-benefit analyses. 

This third cost guide was developed specifically for use by 
piogram evaluatocs in local education agencies. It describes 
several techniques that can be used to "streamline," or expedite, 
the steps of a cost-outcome study. The guide focuses primarily 
on educational program evaluations, but the methods described are 
applicable to any evaluation conducted for internal decision- 
making. 



RATIONALE FOR STREAMLINING 

Guides No. 2 and No. 4 describe the steps of traditional 
cost-ootcome studies for state and local educational evaluations 
as advocated by economists, cost-analysts, and evaluators. The 
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assumptions and emphases of the two guides are sijnilar to those 
of existing texts. This third cost guide differs from the 
previous two in its eirphasis on practicality, it suggests ways 
to streamline the conduct of a cost-outcome study e 

Streamlining involves the modification of cost-analysis 
methods to make them more appropriate for the limited time and 
resources, and immediate information needs, of local- level 
program evaluations. In comparison to traditional formal 
methods, streamlined methods can: 

• increase the likelihood of cost analysis methods 
being used by program evaluators in local education 
agencies; 

• increase the interpretability of study results at the 
local level; and 

• provide more immediately useful information to local 
decision makers, while sacrificing generalizability 
to other settings. 

While streamlining has several advantages over more formal 
cost-analysis methods, it would not be useful unless the 
reliability and validity of study results are preserved. The 
suggestions outlined in this guide preserve these standards. 
They increase local usefulness by shifting the focus to the 
immediate program setting, away frc»n concerns about generalizing, 
to other populations or settings. Therefore, results of such 
studies are not appropriate for distribution to other programs, 
since there is a danger of misinterpretation and 
over-generalization. 

Because the concept of streamlining is new, some explanation 
for its development is warranted* The following sections discuss 
the traditional view of cost analysis, the orientation of 
existing texts on the subject, current applications of cost 
analysis methods for program evaluation, and how streamlining can 
improve the reliability and validity of cost-analysis studies for 
local program evaluation. 

The Basis of Traditional Cost-Analysis 

Up to this time, many program evaluators have viewed cost 
analysis as a time-consuming and technical technique ;]ustifiable 
for only major studies. This view is supported by most existing 
texts, which describe those cost-analysis methods best suited for 
answering societal- level questions. For example. Levin (1983) 
discusses the measurement o£ "opportunity costs,* i.e., all costs 
to the program — monetary and non-monetary — to determine *the 
value of the sacrifices made by society — the value of what must 
be given up— to undertake an intervention* (p. 80). This concept 
of societal sacrifice is important for the evaluation of major 



public interventions* However, it is less impcrtant for small, 
informal evaluations. Moat available texts are based on this 
concept of societal or public-welfare criteria (e.g., Thompson, 
1980, or Sugden 6 Williams, 1978). 



The orientation of Existing Texts of Cost Analysis 

Many texts make the assumption that societal* level questions 
can lessen the usefulness of cost analysis to program 
evaluators. Table 1 illustrates this lack of accord by com^^ring 
the assumptions of most current cost-analysis texts to the 
conditions mder which most cost analyses are conducted in local 
program evaluations. 



Table 1 



Differences Between Text Assumptions and 
Practicing Conditi(xis 



Text Assumptions 

Based on public welfare 
National or International 
level 

Large-scale public policy 
Evaluates 2^ progreuna 
Experimental design 
General izability is 
important 



Practicing Cr.nditions 

Based on sponsor needs 
Organizational level 

Small scale prograun evaluation 
Evaluates 1+ progr2un(8) 
Quasi-experimental design 
Generalizability is not 
important 



Tabl.! 1 makeb clear the reasons why existing texts have been 
of limited usefulness to district-level evaluators^ These 
evaluators conduct studies to answer program-specific questions, 
often designed around a single program* No;v-general izability of 
results is usually not a problem, since the results of the study 
need not be applicable beyond the specific program. Decision 
makers want only a credible answer to their evaluation questions. 



Current Applications of Cost Analysis 
Methods in Program Evaluation 



Recent studies of applications of cost analysis methods at 
the state level (Smith & Smith, 1984) and local level (Smith, 
1984) show that many educational evaluators a. currently 
conducting some form of cost analysis, and many will be sharply 
increasing their use of the procedure over the next five years. 

An examination of the types of studies currently being 
conducted, however, shows that they tend to be of a "home-grown" 
variety rather than reflective of the formal procedures 



prescribed by economists (Smith & Smith, 1984) • Prograun 
evaluators tend to use simple analysis methods, such as cost 
descriptions of a single program, rather than a comparison of the 
cost-outcome ratios of two or more programs. It is not 
surprising that the methods used in local evaluations differ from 
those methods advocated by current texts: Because the local cost 
analysis studies are conducted under a short timeline, the 
questions addressed are of a highly specific nature, and the 
results are not intended for use outside the program. 



The Case for Streamlined Methods 

Pitz and McKillip (1984) have presented a cogent argument 

against the use of traditional cost-analysis methods for program 
evaluation* They contend that methods based on the concept of 
public velfare rather tlian on the values of the decision makers 
may not provide useful evaluation information. Most cost 
analysts agree that the study design must be tailored to best 
meet the needs of the decision maker. For example, Thompson 
(1980, p. 105) writes that in order "for a benefit*cost analysis 
to serve decisionmakers best, it must accurately reflect their 
values." Similarly, Levin (1983) directs the reader to assess 
the needs of the primary audience prior to selecting an analysis 
method. However, the goal of meeting the needs of the primary 
audience can get lost if one adheres strictly to a formal system. 

Streamlining advocates a thoughtful rather than a restrictive 
approach to the conduct cf, a cost-outcome study, with analysis 
strategies chosen because of their local applicability. Most of 
the streamlining techniques suggested in this guide are also 
mentioned in the traditional cost-analysis texts, but they are 
difficult tc isolate from the technical detail required for a 
societal-level study. 

An example is the technique of excluding the cost of "free" 
resources such as volunteers or donated equipment which is 
suggested by Levin (1983). This guide presents many such 
recommendations, with an overriding perspective on emphasizing 
practicality. Examples of questions that could be profitably 
addressed with streamlined methods are as follows: 

Which was the most cost-ef fecive reading progreun in the 
district this year — peer tutoring or computer-assisted 
instruction? 

What were the costs and effects of supplemental math 
instruction for fifth-grade students? 
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The Effect of Streamlined Methods on the Reliability 



and Validity of District-Level Studies 



It is important that streamlining not compromise the 
reliability and validity of cost analysis procedures. With 
careful implementation, these concerns can be successfully 
addressed. In fact, it is often the haphazard and careless 
implementation of the comprehensive, formal, highly technical 
procedures that impair reliability and validity. Therefore, in 
some local settings reliability and validity will be enhanced by 
streamlining. 

Consider the reliability issue first. If the recommended 
procedures are poorly understood or beyond the scope of local 
resources, program- level cost analyses will probably not be 
conducted in a systematic manner. The techniques used may be 
based on local conditions such as availability of personnel or 
pre-existing data that ore not entirely appropriate. Hasty 
estimation of financial figures may be substituted for careful 
calculation. 

in contrast, the provision of a sensible, systematic approach 
to cost analysis will increase the likelihood of reliable data 
and accurate documentation. A study conducted using a systematic 
method can be replicated in later years for comparison purposes. 

With regard to validity, streamlined methods will increase 
the meaningfulness of the results to the personnel who use them. 
It has already been argued that the traditional methods, as 
described in available texts, are often not appropriate for local 
decision making. Haggart (1978) argues that the calculation of 
costs using methods which emphasize generalizability of results 
do not provide an accurate description of actual progr2uii costs. 
Accurate accountings of such program costs are needed if a study 
is to be valid for decision makers. 



STRBMILINIIIQ TECHNIQUES FOR 
MBASURING PROGRAM COSTS 

There are two stages of a cost-outcome analysis: measurement 
of program costs and measurement of prograun outcomes. The rest 
of this guide focuses primarily on these st^s. It is assumed 
that the reader is familiar with Guides No. 2 and No. 4 in this 
series, so that specific steps for determining costs i d outcomes 
will not be restated here. The discussion will be limited to 
streamlining techniques. 

The two steps given in Guide No. 2 for figuring program costs 
include: 

• listing program I'esources, and 

• calculating the costs of those resources. 




To streamline the calculation of prog.ara exeats ^ an 
intermediate step can be added: 

• identifyinq resources that represent direct cost to 
the sponsor • 

If the evaluation question pertains to the direct cost 
assumed by the sponsor, the calculation of program cost should 
include only those resources that involve direct costs. Let's 
look at each of these steps. 



Step 1: Listing Progreun Resources 

The identification and detailed description of all program 
resources is critical to the conduct of a valid cost analysis. 
This step requires substantial investigation by the evaluator. 
(As described in Guide No. 2, the budget should not be used in 
this step except as to verify that no resources were omitted.) 

The goal of this activity is to provide a written description 
of all program resources, including such topics as the job 
descriptions of all paid and volunteer staff, and the activities 
of parents who must transport students to and from the program. 
An observer considering adoption of a program should be able to 
tell from the listing of resources exactly what it would take to 
start up or operate the program. This step is the most important 
aspect of the entire costing procedure. 



Step 2; Identifying Resources Representin g 
Direct Cost to the District 

in most cases, the decision maker wants to know what it costs 
to start up or operate the program (see G^ide No. 4 for a 
discussion of the differences between stai.t up and operation 
costs) • This generally does not require the calculation of the 
value of resources such as volunteer time. For instance, while 
it is important to know that the program requires 6 hours of 
students' time and one hour of parents' time per week, the 
monetary value of this time is relatively unimportant, since it 
is not a direct cost the sponsor must absorb. 

Traditional cost-analysis methods/ which assess the 
opportunity costs of a program, would require that the monetary 
value of the parents' and students' time oe calculated using 
econonetric methods. All these costs would then be included in 
the {MTogram total cost. Such inclusion of opportunity costs 
would spuriously elevate the total cost figure for local analysis 
purposes. 

The cost total should only include those resources that 
currently require district funding, or are expected to do so in 
the future. Following are two factors in identifying which 
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rasources ^ald be included in the total cost figure: 



!• the parameters of the program; 

2. the reliability of the *free" resources. 

First, consider the parameters, or scope, of the program wnen 
deciding which resources to include in the cost figure. If a 
program is an adjunct to regular school programming, then only 
those costs over and above the regular school costs need to be 
included. For example, a reading program that pulls students cut 
of regular class for one hour a week is an adjunct to regular 
instruction. Only the cost of the pull-out progreun needs to be 
calculated, not the cost of running the school building (e.g., 
electricity, maintenance) during that hour, since it would be 
open in any case. Although space and furniture are being used, 
they are part of regular school equipment and their cost is 
"sunk," that is^ paid for out of a general account by the 
district regardless of use. 

You could also compare the list of resources, crossing out 
the common ones for both programs. Those resources used in both 
programs, such as space, equipnenv or instructors (if their 
salaries were comparable) will increment the total cost by 
equivalent amounts, and so could be cancelled out. This is 
called a marginal cost analysis and is described by Levin (1983) 
and Thompson (1980). For most school program evaluations, a 
marginal cost analysis is probably quite appropriate, and it 
greatly simplifies the figuring of program costs. (Returning to 
the previous caution, however, it is apparent that this procedure 
limits the generalizability of cost estimates to other settings.) 

The second factor involved in the total cost figure is the 
cost of resources that are not paid for with program or district 
funds (such as volunteer time or donated equipment) . These may 
not need to be figured. For each such resource, the evaluator 
should determine whether the resource will continue to be 
available at no charge. If there is reason to believe that a 
free resource will not be available for the duration of the 
study's time span, then the cost of that resource should be 
included in the total cost figure. If, however, the resource 
will continue to be available at no cost, it need not be 
included. Following are scne typical resources that may not 
involve direct cost to the sponsor: 

• building rental; 

• borrowed, inherited, or donated equipment; 

• equipment paid for in previous ye^^ts; 

• student, parent, or volunteer tiibe; 
m 8tudent*purchased equipment; 

• inservice training provided by external agencies. 

The final report should contain a listing of all program 
resources, whether or not they are included in the total cost 
figure. A separate listing of all resources excluded from the 



cost total should also be provided, along witii a rationale for 
the exclusion of each resource. 



Step 3; Determining the Cost of Resources 

use of Market and Shadow Pricing . Each resource remaining on 
the list after the "no-cost" items have been checked off must be 
assigned a cost. Guide No. 2 in this series described the use of 
market-pricing and shadow- pricing techniques for valuing 
resources, in market pricing^ the open-market value of a 
resource is ured as a cost estimate. In shadow pricing, the 
open-market value of a similar or equivalent resource is used. 

For the vast majority of program- level resources, market 
pricing is used. Shadow pricing is helpful when the resource is 
no longer a market coomodity, as might be the case with a 
discontinued microcomputer model. However, program resources for 
which shadow- pricing techniques would be needed probeibly no 
longer represent a cost to the sponsor. In this event, the 
resource need not be included in the program cost figure. 

The budget can also be used to obtain a limited amount of 
information about the cost of some resources. While there are 
reasons against using the budget to determine program coBts, it 
can provide useful information for seme resources. For instance, 
the salary of personnel (including benefits) can be obtained. 
The evaluator should carefully consider the accuracy of each 
budgeted figure before incorporating it in the cost analysis. An 
example of a worksheet that incorporates each of the three steps 
in the tabulation of program costs is provided in the Appendix. 

Figuring Present and Annualized Values 

When the time span of an evalua^.ion exceeds a year, it is 
necessary to determine tae present and annualized values of some 
resources. Guide No. 4 e> ains how to calculate these figures. 
For local-level evaluat.Lo^^ however, if only the direct cost of 
the program to the spons"* for c year is requested, then 
present and annualized va'ues ne. not be figured. This 
treatment of program costs will contribute to a distorted view of 
typical program costs (e.g., the estimate of program cost may be 
significantly elevated due to the purchase of equiixuent) . But it 
may address the decision maker's immediate information needs. An 
explanation should be provided in the evaluation report about the 
treatment of resource cost estimates where calculation of present 
or annualized values might be warranted. 
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Conducting a Sensitivity Analysis 



You Bay have guessed that estimating program cx>st is a 
tentative procedure* The accuracy of cost estimates can vary due 
to many factors. For example, what if a no-cost resource 
suddenly becomes a cost resource? Perhaps the current 
microcomputer will need to be replaced with a new one, or will 
need extensive repair • It is important to talk with program 
staff to obtain their judgments about the stability of program 
resources. As another exeunple, there may be uncertainty about 
the reliability of the cost estimate (e.g., the new microcomputer 
will cost from $5,000 to $8 ,000). In such cases, traditional 
cost analysts recommend a sensitivity analysi. 

A sensitivity analysis can be conceptualized as a best 
case/worst case scenario. In the microcomputer example, the dest 
case would involve the microcomputer continuing to function with 
no necessary repairs for the rest of the yeeu:. In the worst 
case, the microcomputer might need to be replaced, which could 
increase the program cost by $5,000 to $8,000. 

The lowest possible costs for all resources with uncertain 
values should be summed to provide an overall best case. 
Similarly^ the highest possible costs should be summed to provide 
an overall worst case. These scenarios, along with a central 
tendency value of possible costs, should be provided to the 
decision makers. 



A Note of Caution 

When implementing these suggestions, the previously stated 
questions need to be borne in mind. For instance, the addition 
of one large one-time cost for a pro9«.£un resource to a one-ye2u: 
cost estimate could be misleading if the study results were used 
to estimate program costs in subsequent years, or if the results 
were disseminated to other districts as a representative 
description of program costs. Such misuses of the data aire to be 
care felly avoided. 

If streamlined methods are to make a positive contribution to 
program evaluation methodology^ the use of the streamlined 
methods and the resultant data must be relevemt to the specific 
evaluation question. These methods are entiraly appropriate wh^n 
used judiciously. 



STBEANLINING TBCJNIQUBS FOR MEASURING PROGRAM OUTCOMES 

This section looks at methods for streamlining the 
measurement of program outcomes. Guide No. 4 looked at the 
identification and measurement of outcome veuriables and explained 
how to interpret cost-outcome (e.g., cost^utility. 
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cost-effectiveness, and cost-benefit) ratios. Again, for tne 
sake of bre^rity, these will not be restated here. 



This section is brief because suggesting methods of 
streamlining the assessment of prograun effectiveness to 
evaluators is like suggesting methods of streamlining the 
assessment of program costs to economists; that is, knowing the 
most efficient methods of collecting effectiveness data is the 
evaluator's stock in trade. The purpose of this section is to 
review a few streamlining techniques for outcome measurement 
practices \'Jiat most evaluators have used for years. 



Usirg a Quasi'^Exper imental Design 

Although Guide No. 4 is presented as compatible with texts or; 
traditional cost analyF methods, it does differ from existing 
text; in one signiZicaf.. way it recognizes the use of 
quasi- dxp^rinentaJ research designs. The guide u'scusses 
possible effects on the results due to non-random assignment, 
both of participants to progrzuns and of programs to treatments. 
In many cases, quasi-experimental designs are not luxuries, but 
necessities for the completion of evaluations. When used with 
appropriate safeguards, they can be viewed as streamlining the 
assessment of program outcomes. 



Evaluating a Single Program 

Another assumption held by traditional cost analysts is that 
the methods are always conducted for comparative purposes. The 
reason for this a ^sumption is understaniable: a cost-outcome 
ratio is un inter pretable until it is compcired with another 
ratio. Guide No. 4 provides an example of a cost-effectiveness 
analysis of two reading progr£uiis. The cost-effectiveness ratio 
of one program was 7 to 1. In other words, it cost $7.10 to 
produce each point iijprovement on a reading test. The figure has 
little intrinsic meaning: it is meant to be used in a 
coflq>eur ison • if the cost-effectiveness ratio of a second program 
was 2, the ratios can be interpreted. The $7.10 per point in the 
first case is seen to be much less cost effective than the $2.0C 
per point in the second. 

Despite the problem of interpreting a single cost-outcome 
ratio, many evaluations involve only one program. There are 
ways, however, to provide comparisons without evaluating two 
discrete programs. 7or axeunple, an evf^iuation of peer tutoring 
could be conducted by comparing peer-tutoring at the fifth gra-^e 
with peer- tutoring at the sixt ^ grade. In addition, cost-outcome 
data collected ir ^e year could be used during that year for 
program management:, then compared to a replication of the study 
in the following year. There is also the case study approach, 
which provides contextual detail on cost data for management 
purposes, and on outcooee data for evaluation purposes. 
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Typett of Cost-Analysis Methods 



Ttiere are four cost-a^^alysis methods: cost-feasibility, 
cost-utility, cost-effectiveness, and cost-benefit. Each method 
can be described as follows: 

Cost-Feasibility . Cost-feasibility is, by definition, the 
most streamlined of the cost analysis methods because it doe3 not 
involve assessment of program outcomes. As a result, it also 
does not require comparison with another program in order to be 
meaningful. An examination of the cost data for a single program 
can be very helpful for program managers. The type of 
information that this kind of examination can provide is 
discussed in Guide No. 4. 

Cost-Utility . Cost-utility analysis involves the subjective 
valuing of probably program outccxces, and does not require 
collection of empirical outcome measures. A frequent criticism 
of this method is that it is too subjective, and therefore an 
unreliable measure of progra^a value. However, the systematic 
application of the method rinderlying cost-utility analysis — 
multiattr ibute utility analysis — can provide information that is 
very useful to decision makers, since it systematically 
aggregates and summarizes their own values. 

In general, cost-utility analysis is the most underrated of 
all cost analysis methods. Pitz and NcKillip (1984) have 
recently come out in strong support of its use in decision 
making. Guir*^ No. 4 discusses how to conduct a cost-utility 
2Uialy8is . 

Cost Effectiveness . Cost-effectiveness analysis has been 
widely recognized as the method of choice for social service 
program evaluation. It requires the collection of traditional 
effectiveness data» and as such is second- nature to program 
evaluators. G^'de No. 4 provides a listing of examples of 
eff' jtiveness data typically used by evaluators. 

Host evaluators are familiar with methodological approaches 
to streamlining, such as aampling program participants ^ or using 
instroments with established reliability and validi'i.y. In 
addition, the evaluator may want to consider u^in^ che data 
collected for different purposes, such as data f^'c statewide 
testing, or data required for federall;/ funJe^^ programs such as 
Chapter 1 programs. 

The evaluator could obtain access to available effects data 
and reconstruct the cost data retrospectively. If care is taken 
to prnerve accuracy, this procedure can provide information that 
is very useful to managers and dec is ion- makers. Therefore, the 
conduct of a co8t-*effectiveness analysis does not necessarily 
require the laborious collection of data. 
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Cost-Benefit , This analysis, which places a monetary value 
on program outcomes, is commonly mistaken to be synonymous with 
cost analysis in general. Tnis is unfortunate, because 
cost-benefit analysis is tho most difficult technique to conduct 
correctly, and is the least meaningful method for most social 
service program evaluations. There are very few cases where 
cost-benefit analysis would be chosen for program evaluation 
because most social service programs are simply not amenable to 
this analysis. 
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ANNOTATED BIBLIOGRAPHY 



Andersen, F. (1982). Problems in estimating the costs of 
special education in urban areas- The New York City case. 
vTournal of Education Finance y 403-424 • 

Tnis article provides an example of the tabulation of special 
education progreun costs, and includes a thorough discussion 
of the calculation of costs such as marginal and "displaced" 
costs. It also provides information about conceptual issues 
in cost analysis, such as uncertainty in defining the scope 

of services and working in the context of a political 
environment. 

Catterall, J, (1982}. The costs of school testin^j progreuns (CSE 
Report No. 194). Los Angeles, CA: University of California 
Center for the Study of Evaluation. 

Catterall provides a uniquely thorough assessment of 
nonmonetary as well as monetary costs of testing progreuos. 
This report looks at the importance of assessing opportunity 
costs from the perspective of the students, the teachers, and 
the school district. The report provides an excellent 
discussior of the types of costs to consider in a complete 
cost analysis. 

Dorr-Bremme, D., Burry, J., Catterall, J., Cabello, B. 

& Daniels, L. (1983). The costs of testing in American 
public schools (CSE Report No. 198) . Los Angeles, CA: 
University of California Center for the Study of Evaluation. 

This report focuses on the assessment of the costs of testing 
in a case report format. Three case studies are described. 
Detail is provided on all aspects of the process of 
conducting a cost analysis, including idercif ication of 
resources, collection of cost data (e.g., use of survey and 
ii^terview techniques) , and selection of cost analysis 
method. No outcome data was collected for this study. 

Haggart, S. et al. (1978). The resource approach to the analysis 
of educational project coat (SN 017-080-01914-1). 
Washington, D. C: U. S. Government Printing Office. 

Haggart and her colleagues have developed a method for 
collecting and valuing educational p^ am costs. The most 
valueUDle aspect to this monograph is the authors* distinction 
between district- specific and program-cccnparable costs. They 
contend that methods for estimating costs differ according to 
the use of the cost data. If data are to be used only within 
a particular district, then absolute district costs can be 
used as the cost estimate. If however the cost data are to 
be distributed beyond the district, then alternative, 
theoretical, and generalizable cost-valuing methods are 
^ appropriate. 

ERLC 

™ 17 



Kosier, K. (1984, October). Cost effectiveness analysis of 

alternatives for Chapter 1 reduction . Madison, WI; Madison 
Metropolitan School District. 

This paper does not describe how cost data were collected* 
and it appears as though budget data were used. However, it 
does provide an interesting example of how cost-effectiveness 
ratios were used to help decision makers decide which program 
aspects were most and least effective. For example, Kosier 
calculated cost-effectiveness ratios across grades, 
achievement level, lunch programs, and aide-teacher 
combinations. The paper shows what comparisons can be made 
and how the cost-effectiveness ratios can be interpreted to 
assist decision makers. 

Levin, H. M., Glass, G. V., Meister, G. R. (1984). 

Cost-effectiveness of four educational interventions (Report 
No. 84-All) . Stanford, CA: University of California 
Institute for Research on Educational Finance and Governance. 

Levin and his colleagues use meta-analysis to look at the 
cost-effectiveness of four math an<l reading interventions: 
cross-age tutoring, computer-assisted instructions, reduction 
in class size, and increases in daily instructional 'uli*:e. 
The meta-analysis methods of aggregating outcomes Kill 
probably not be used by most practitioners. However, v^e 
identification emd tabulation of costs is thorough and oould 
be helpful as a model. In addition, the results of the 
analyses may be used as a meaningful comparison for 
interpreting local results. 

Levin, H. M. (1983) . Cost-effectiveness; A Primer . Beverly 
Hills, CA: Sage Publications- 

This text is probably the best available resource for the 
evaluation practitioner learning about cost-effectiveness 
analysis. It provides a complete description of how to 
figure prograun costs and includes sufficient technical detail 
to enable the reader to value his/her own resources. The 
value of this book is that it discusses cost-effectiveness 
analysis at the program level, therefore providing examples 
that are meaningful to practitioners. The text does fall 
short in detail about defining and collecting outcome data 
for utility, effectiveness, and benefit studies. The 
inexperienced i'eader would generally have to consult with 
additional resources for explanations about these steps. 
This book is an essential first-reader for evaluators looking 
for information about cost analysis methods. 




16 



8 



Thompson, M. S. (1980). Benefit-cost analysis for program 
evaluation , Beverly Hills, CA: Sage Publications. 



This thorough, detailed text tells how to conduct policy-level 
cost-analysis studies. It provides information about the 
data that would have to be collected in order to conduct 
"public" cost analyses, and therefore goes beyond the needs 
of most evaluation practitioners. Although it is a good 
reference to have on hand, it is probably not a text that the 
evaluator would want to read from cover to cover. 



APPENDIX 



Worksheet Showing the Three Steps of 
Tabulating Program Costs 



(1) P«rflonn«lt 

coordinator {T/T) 

tmMChn aid* (T/T) 
voluntMT (HA) 



Cost to 

District 



30,000 

10,000 
0 



Explsnstory Coient* 



budgst fiquzm, includss 

b«n«f its 
bud9«t figure 



(2) rscilitisst 
clssarooB 

furniturs 



district ovsrhsad 
dis'crict ovsrhsad 



(3) Bquipaanti 

10 Appla lla coaputara 0 
■aintananca 5 ,000 



puxctiasad U at yaar 
based on last year's 
figure 



(4) Hatariala « Suppliaj: 

aoftwara 4 ,000 

supplias 0 



aatlAata obtainad froa 
dealer 

ainlAsl, so not included 



(5) Miscellaneoua: 
insurance 

inservice training 



district oyarhead 
district overhead 



(6) Clientt 0 

tranapoctation 0 

coaputar diaka o 

2 bra. parents' tlae 0 

6 bra. atudentfl tiae 0 
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